Support Vector Learning for Fuzzy Rule - Based Classi cation Systems

|To design a fuzzy rule-based classi cation system (fuzzy classi er) with good generalization ability in a high dimensional feature space has been an active research topic for a long time. As a powerful machine learning approach for pattern recognition problems, support vector machine (SVM) is known to have good generalization ability. More importantly, an SVM can work very well on a high (or even in nite) dimensional feature space. This paper investigates the connection between fuzzy classi ers and kernel machines, establishes a link between fuzzy rules and kernels, and proposes a learning algorithm for fuzzy classi ers. We rst show that a fuzzy classi er implicitly denes a translation invariant kernel under the assumption that all membership functions associated with the same input variable are generated from location transformation of a reference function. Fuzzy inference on the IF-part of a fuzzy rule can be viewed as evaluating the kernel function. The kernel function is then proven to be a Mercer kernel if the reference functions meet certain spectral requirement. The corresponding fuzzy classi er is named positive de nite fuzzy classi er (PDFC). A PDFC can be built from the given training samples based on a support vector learning approach with the IF-part fuzzy rules given by the support vectors. Since the learning process minimizes an upper bound on the expected risk (expected prediction error) instead of the empirical risk (training error), the resulting PDFC usually has good generalization. Moreover, because of the sparsity properties of the SVMs, the number of fuzzy rules is irrelevant to the dimension of input space. In this sense, we avoid the \curse of dimensionality." Finally, PDFCs with di erent reference functions are constructed using the support vector learning approach. The performance of the PDFCs is illustrated by extensive experimental results. Comparisons with other methods are also provided. Keywords|Fuzzy systems, statistical learning theory, support vector machines, fuzzy classi er, kernel methods, pattern classi cation.